This invention relates to a man-machine interface and in particular to an interface that measures body part positions and provides feedback to a user""s hand and arm.
A new manner of computer interaction is now in its infancy. The words xe2x80x9cvirtual environmentxe2x80x9d or xe2x80x9cvirtual realityxe2x80x9d will soon be commonplace. A virtual environment is an environment where some portion of the environment is artificially simulated, most often via a computer. A computer may create a graphic simulation of an environment, complete with graphic images of chairs, windows, doors, walls, etc., and even images of other people. The computer may also simulate environmental sounds. The generated objects may be viewed on a common two-dimensional display, such as a computer screen, or, by viewing with special stereoscopic equipment, the objects may be made to appear three dimensional.
The most natural way for an individual to interact in a virtual environment is to directly control a graphical representation of himself. For example, if the individual turns his head, the display screen at which he is looking is appropriately updated. Also, if the individual reaches out and closes his hand, the computer generated image of his hand on the screen reaches out and closes. Such virtual environments have been discussed in the literature.
To create the sensation of virtual reality, the computer should be able to generate and manipulate graphic images of real or imaginary objects in real time. Although generating a graphic representation of an environment may be time consuming and non-trivial to implement, much of the theory has been explored and is well-understood by those skilled in the art of interactive 3-D computer graphics and solid modeling. The invention described here pertains to the important related area in which relatively little research has been done, i.e., xe2x80x9cHow may a human user perceive grasping force and from his computer-generated counterpart in the virtual environment?xe2x80x9d
There are many peripheral devices which have been created to allow a user to enter information into the computer. The most notable of these is the standard QWERTY keyboard. Besides the numerous modifications of this xe2x80x9ckey inputxe2x80x9d concept, there are many other devices with their associated permutations. A partial list of such devices includes mice, joy-sticks, trackballs and Computer-Aided-Design (CAD) tablets. The main drawback of these computer input devices is that they don""t permit human users to enter information in a manner which may be the most efficient and natural. For example, in a CAD software program, the human designer may wish to rotate a 3-D graphic representation of a block on a computer screen to view and modify the hidden side. Using currently available input devices, the designer must select the axis or a sequence of axes about which the object must be rotated to achieve the desired orientation and view. After the desired axis is selected, the amount of angular rotation must be determined, usually by the linear motion of a mouse or by entering the desired amount of rotation as a decimal quantity via the keyboard. This whole procedure seems very awkward and non-intuitive when compared to what a person would normally do when confronted with a similar task in the xe2x80x9creal world,xe2x80x9d i.e., he would simply reach out, pick up and rotate the object.
Instrumented gloves which provide finger-position information to the computer have been used to manipulate simulated objects in virtual environments. Such gloves have also been used in telerobotics to control highly dextrous end-effectors to grasp real objects. However, lack of force feedback to the glove wearer has reduced the effectiveness of these open-loop manipulation approaches. Imagine a 3-D graphic model of an egg on a computer screen. Suppose you are wearing a glove which maps your finger and hand motions to a graphic image of a hand on the same screen as the egg. As you move your hand and fingers, the corresponding graphic images of the hand and fingers move in a similar manner. The task is to move your own hand and fingers to control the graphic hand on the computer screen to pick up the egg. To accomplish this task you must provide enough force to reliably grasp and lift the virtual egg, but not so much force such that the egg is crushed. Without some kind of grasping force and tactile feedback, this task would be extremely difficult.
Attempts have been made to provide information about simulated contact with virtual or telemanipulated objects to senses other than the corresponding tactile senses. One method of simulated feedback which has been tested uses audible cues. For example, the computer may beep when contact is made. Another simple method is to highlight the object once contact is made. Both these methods will require the user to re-learn hand-eye coordination. It may be frustrating and time consuming for the user to learn one of these xe2x80x9cunnaturalxe2x80x9d methods of grasping an object, and the sensation of interacting in a virtual environment will be reduced.
More recently, approaches have been developed to directly exert forces to the fingertips. One such approach uses pneumatic pistons located in the palm of the hand to exert resistive forces at the fingertips. The disadvantages of such an approach are numerous. First or all, pneumatic cylinders have low mechanical bandwidth and cannot exert very large forces because the limited workspace of the palm limits their size. Additionally, such actuators tend to be noisy and the fact that they are located in the palm limits the range of motion significantly. Other approaches have used servo-motors located directly on the back of the hand. Such approaches tend to be quite bulky and often need to be supported by robotic arms and thus are not well suited for desktop applications. When robotic arms are not used, hand and arm fatigue are often a problem as it is quite difficult to produce a device that is small and light enough for prolonged usage. Additionally, such devices often do not provide feedback to all the fingers in an effort to minimize bulk. Finally, such devices typically suffer from a limited range of motion which hinders manipulation.
Therefore, it will be appreciated that there remains a need for a man-machine interface for the hand that is capable of sensing finger and hand positions and hand orientation, that provides appropriate force-feedback, and that overcomes the other limitations in the state-of-the-art as described herein before.
One object of the invention is to provide a man-machine interface which may be employed in interactive computer applications. Another object of the invention is to provide a force feedback control system capable of controlling a set force to a selected part of the body, e.g., the fingertip. et another object of the invention is to provide a man-machine interface comprising a glove capable of sensing finger and hand positions and hand orientation, which may exert, measure and dynamically vary and control the forces applied to each finger. Another object of the invention is to provide a digital control system capable of sensing the force applied to the fingertip and capable of using this applied force signal to control the fingertip force to a desired force set point which may vary as a function of finger position. Still another object of the invention is to provide a force feedback system which may be employed in many different applications, such as virtual environments, telemanipulation and interactive 3-D graphics, telerobotics and Computer Aided Design (CAD). Yet another object of the invention is to provide more natural and intuitive feedback during object/environment interaction.
The subject invention introduces new techniques for providing grasp force feedback and grounded force feedback to the hand of a wearer. The feedback techniques are largely predicated on transmitting a force from a remotely located actuator to the site of force application via a tendon-in-tendon-guide structure. Various tendon/tendon guide structures are provided, some comprising flexible tendon guides and some comprising rigid tendon guides. In one useful embodiment of the subject invention, the tendons are routed over a series of moment-augmenting structures on the dorsal surface of the hand, where the structure determines the level of moment applied to joints of the hand for a given fingertip force. The structure is typically designed such that a larger moment is applied to the metacarpophalangeal joint than a joint more distal. In another useful embodiment, 5- or 7-bar linkages are used to apply force only to the fingertip relative to a location typically either on the back of the hand or a structure supported by a ground-referenced robotic arm. When used with a ground-reference robotic arm, grasp-force devices become lightweight, low-inertia ground-referenced force-feedback devices.
In one aspect, the inventive structure provides apparatus for attachment to a body where the body has a sensing body link connected to a non-sensing body link with at least one sensing body joint between the sensing and non-sensing body links. The apparatus includes means for applying force to the sensing body link, attachment means for attaching to the means for applying force and to the non-sensing body link, and means for generating a force at the sensing body link and a moment at the sensing body joint. The apparatus also includes means for applying the generated force between the sensing body link and the non-sensing body part. In one embodiment of the inventive structure, the means for applying the generated force includes a moment-augmenting structure (such as for example, towers and cams). The inventive structure also includes a tendon elevated by the moment-augmenting structure, where the tendon is connected at the force-applying means at one end, and to the force generating means at the other end; and tendon guiding means for guiding the tendon between the force-applying means and the force generating means.
In one particular embodiment, the moment-augmenting structure comprises first and second elements connected by an articulated link such that the two elements move in the same plane. In another embodiment, the moment-augmenting structure comprises a composite member of some complexity comprising a flexure-articulating component and an abduction-articulating component, the composite member further comprises two revolute joints, wherein the flexure-articulating component is attached to the abduction-articulating component by one of the revolute joints and rotates relative to the abduction-articulating component, and the abduction-articulating component is attached to the attachment means at the non-sensing body link by means of the other one the revolute joints. In still another embodiment, the moment-augmenting structure comprises a simple member including means for attachment to an intermediate link between the sensing and non-sensing links and a tendon-elevating guide connected to the attachment means.
In another embodiment, the force-applying means includes a platform displaced from the sensing body link when in an unactivated position and in contact with the sensing body link when in an activated position. In still another embodiment, the apparatus may include a second force generating means connected to the apparatus for providing force to the apparatus relative to a reference point off the body.
In one embodiment of the inventive method for use in a device for attachment to a body having a sensing body link connected to a non-sensing body link, includes the steps of: applying force to said sensing body link; attaching the body to the force-applying means and to said non-sensing body link; generating a force at the sensing body link and a moment at said sensing body joint; and applying the generated force between the sensing body link and the non-sensing body part; the step of applying the generated force comprising applying the force via a moment-augmenting structure and a tendon elevated by the moment-augmenting structure, where the tendon is connected to receive the generated force at one end and to apply the applied force at the other end; and guiding the tendon between the force-applying means and the force generating means.
A control system and method that senses the force applied to the fingertip; and controls the fingertip force to a desired force set point in response to the sensed applied force signal, where the desired force set point may varying as a function of finger position, is also described.
In one aspect, the invention provides for the use of a flexible housing which may comprise one or more concentric flexible casings which guide a force-transmitting flexible elongated element such as a flexible, low friction/stiction, high modulus of elasticity thread or a shape-memory alloy wire which serves as a tendon and is used in tension to apply force to a sensing body part. In another aspect, the invention provides for the use of force actuators to generate force which is transmitted to the sensing body part via flexible tendon cables, or pneumatic or hydraulic tubes, and used by a force applicator to apply force to the sensing body part. In still another aspect, the invention provides for the use of a support to which the flexible tendon cables or tubes are secured. The support may be a reinforced wrist-strap when the sensing body part is part of the hand. In yet another aspect, the invention provides for the use of a mechanical structure to augment the mechanical moment and which is attached to the back of the hand to route force applying tendons to each of the fingertips without hindering hand movement and exerting resistive forces at the fingertips as well as resistive torques at the finger joints. In a further aspect, the invention provides for the use of a pressure, tension and/or force sensor to measure the force applied to the force-sensing body part by the force actuator.
Additional objects, features, and advantages of the inventive system, apparatus, and method will be more readily apparent from the following detailed description and appended claims when taken in conjunction with the drawings.